Draft

OGC Engineering Report

2023 Open Standards and Open Source Software Code Sprint Summary Engineering Report
Gobe Hobona Editor Joana Simoes Editor Tom Kralidis Editor Martin Desruisseaux Editor Angelos Tzotsos Editor
OGC Engineering Report

Draft

Document number:23-025
Document type:OGC Engineering Report
Document subtype:
Document stage:Draft
Document language:English

License Agreement

Use of this document is subject to the license agreement at https://www.ogc.org/license



I.  Executive Summary

The mutually beneficial relationship between open standards and open source software has been instrumental in the update of geospatial technologies across a wide variety of industries. Whereas open source software has enabled many organizations to rapidly implement open standards, open standards have enabled open source software to support a wide variety of use cases through efficient integration and greater interoperability. An enduring challenge however, has been how and when to bring developers from the open standards and open source software communities together.

From the 25th to the 27th of April 2023, the Open Geospatial Consortium (OGC), the Apache Software Foundation (ASF), and the Open Source Geospatial Foundation (OSGeo) held their third joint Open Standards and Open Source Software Code Sprint. The code sprint served to accelerate the support of open geospatial standards within the developer community.

The code sprint brought together developers of Open Standards, Open Source Software and Proprietary Software into the same physical space. Such face-to-face collaboration had been made possible by the support of Ordnance Survey, the event’s Gold-level Sponsor, and the support of Camptocamp, the event host, as well as co-sponsors of the catering namely HEIG-VD (School of Engineering and Management), EPFL ,University of Lausanne , State of Neuchâtel , State of Vaud and Camptocamp.

Previous code sprints had been completely virtual events. Therefore this third Open Standards and Open Source Software Code Sprint held special significance in that it was the first hybrid event (supporting both face-to-face and remote participation) that had been jointly organized by the three organizations. This Engineering Report (ER) summarizes the main achievements of the code sprint.

The OGC is an international consortium of more than 500 businesses, government agencies, research organizations, and universities driven to make geospatial (location) information and services FAIR — Findable, Accessible, Interoperable, and Reusable. The consortium consists of Standards Working Groups (SWGs) that have responsibility for designing a candidate standard prior to approval as an OGC Standard and for making revisions to an existing OGC Standard. The sprint objectives for the SWGs were to:

The Open Source Geospatial Foundation (OSGeo) is a not-for-profit organization whose mission is to foster global adoption of open geospatial technology by being an inclusive software foundation devoted to an open philosophy and participatory community driven development. The foundation consists of projects that develop open source software products. The sprint objectives for OSGeo projects were:

The Apache Software Foundation (ASF) is an all-volunteer community comprising 815 individual Members and 8,500 Committers on six continents stewarding more than 227 million lines of code, and overseeing more than 350 Apache projects and their communities. The sprint objectives for ASF projects were:

The code sprint provided an environment for development and testing of prototype implementations of open standards, including implementations of draft and approved OGC Standards. Further, the code sprint provided a collaborative environment for developers to fix open issues in products, develop new features, improve documentation, improve interoperability with other libraries/products, and develop prototype implementations of OGC standards. Furthermore, the code sprint also enabled the participating developers to provide feedback to the editors of OGC standards.

The code sprint therefore met all of its objectives and achieved its goal of accelerating the support of open geospatial standards within the developer community.

The engineering report makes the following recommendations for future innovation work items:

The engineering report also makes the following recommendations for things that the Standards Working Groups should consider introducing support for:

II.  Keywords

The following are keywords to be used by search engines and document catalogues.

OGC, OSGeo, ASF, Apache, hackathon, code sprint, standards, geospatial, API, open source

III.  Security considerations

No security considerations have been made for this document.

IV.  Submitters

All questions regarding this document should be directed to the editor or the contributors:

NameOrganizationRole
Gobe HobonaOpen Geospatial ConsortiumEditor
Joana SimoesOpen Geospatial ConsortiumEditor
Angelos TzotsosOpen Source Geospatial FoundationEditor
Martin DesruisseauxGeomatysEditor
Tom KralidisMeteorological Service of CanadaEditor
TBATBAContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor

V.  Abstract

The subject of this Engineering Report (ER) is a code sprint that was held from the 25th to the 27th of April 2023 to advance support of open geospatial standards within the developer community, whilst also advancing the standards themselves. The code sprint was organized by the Open Geospatial Consortium (OGC), the Open Source Geospatial Foundation (OSGeo) and the Apache Software Foundation (ASF). The code sprint was sponsored by Ordnance Survey and hosted by Camptocamp.

2023 Open Standards and Open Source Software Code Sprint Summary Engineering Report

1.  Scope

This Engineering Report (ER) summarizes the main achievements of the 3rd annual Open Standards and Open Source Software Code Sprint that was held from the 25th to the 27th of April 2023 in Bussigny, Switzerland. The ER presents the high level architecture, and describes each of the standards and software packages that were deployed in support of the code sprint. The ER also discusses the results and presents a set of conclusions, including recommendations.

2.  Normative references

The following documents are referred to in the text in such a way that some or all of their content constitutes requirements of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies.

Open API Initiative: OpenAPI Specification 3.0.3, https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.3.md

Berners-Lee, T., Fielding, R., Masinter, L: IETF RFC 3896, Uniform Resource Identifier (URI): Generic Syntax, https://tools.ietf.org/rfc/rfc3896.txt

W3C: HTML5, W3C Recommendation, https://www.w3.org/TR/html5/

Schema.org: https://schema.org/docs/schemas.html

Clemens Portele, Panagiotis (Peter) A. Vretanos, Charles Heazel: OGC 17-069r4, OGC API — Features — Part 1: Core corrigendum. Open Geospatial Consortium (2022). https://docs.ogc.org/is/17-069r4/17-069r4.html.

Mark Burgoyne, David Blodgett, Charles Heazel, Chris Little: OGC 19-086r5, OGC API — Environmental Data Retrieval Standard. Open Geospatial Consortium (2022). https://docs.ogc.org/is/19-086r5/19-086r5.html.

Benjamin Pross, Panagiotis (Peter) A. Vretanos: OGC 18-062r2, OGC API — Processes — Part 1: Core. Open Geospatial Consortium (2021). https://docs.ogc.org/is/18-062r2/18-062r2.html.

Joan Masó, Jérôme Jacovella-St-Louis: OGC 20-057, OGC API — Tiles — Part 1: Core. Open Geospatial Consortium (2022). https://docs.ogc.org/is/20-057/20-057.html.

Charles Heazel: OGC API — Common — Part 1: Core, version 1.0.0, OGC 19-072, Open Geospatial Consortium, https://docs.ogc.org/is/19-072/19-072.html

3.  Terms, definitions and abbreviated terms

This document uses the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this document and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.

This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the ‘ModSpec’. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.

For the purposes of this document, the following additional terms and definitions apply.

3.1. API

An Application Programming Interface (API) is a standard set of documented and supported functions and procedures that expose the capabilities or data of an operating system, application, or service to other applications (adapted from ISO/IEC TR 13066-2:2016).

3.2. coordinate reference system

A coordinate system that is related to the real world by a datum term name (source: ISO 19111)

3.3. OpenAPI Document

A document (or set of documents) that defines or describes an API. An OpenAPI definition uses and conforms to the OpenAPI Specification (https://www.openapis.org)

3.4. Web API

API using an architectural style that is founded on the technologies of the Web [source: OGC API — Features — Part 1: Core]

3.5.  Abbreviated terms

API

Application Programming Interface

ASF

Apache Software Foundation

CITE

Compliance Interoperability & Testing Evaluation

CRS

Coordinate Reference System

CSW

Catalogue Services for the Web

EDR

Environmental Data Retrieval

GIS

Geographic Information System

MOU

Memorandum of Understanding

OGC

Open Geospatial Consortium

OSGeo

Open Source Geospatial Foundation

OWS

OGC Web Services

REST

Representational State Transfer

SDI

Spatial Data Infrastructure

TEAM

Test, Evaluation, And Measurement Engine

WCS

Web Coverage Service

WFS

Web Feature Service

WMS

Web Map Service

WMTS

Web Map Tile Service

WPS

Web Processing Service

4.  High-Level Architecture

The focus of the sprint was on the support of implementations of open geospatial standards across various open source software projects. Implementations of approved and candidate OGC Standards were deployed in participants’ own infrastructure in order to build a solution with the architecture shown below in Figure 1.

Figure 1 — High Level Overview of the Sprint Architecture

As illustrated the sprint architecture was designed with the view of enabling client applications to connect to different servers that implement open geospatial standards such as the suite of OGC API standards. The architecture also included several different software libraries that support open geospatial standards and enable the extraction, transformation and loading of geospatial data. The rest of this section describes the software deployed and standards implemented during the code sprint.

4.1.  Approved OGC Standards

4.1.1.  OGC SensorThings API

The OGC SensorThings API provides an open and harmonized way to interconnect devices, data, and applications over the web and on the Internet of Things (IoT). At a high level the OGC SensorThings API provides two main parts, namely Part I — Sensing, and Part II — Tasking. The Sensing part provides a standard way to manage and retrieve observations and metadata from a variety of IoT sensor systems. The Tasking part provides a standard way for tasking IoT devices, such as sensors or actuators. The SensorThings API follows REST principles and uses JSON for encoding messages, as well as MQTT for publish/subscribe operations.

4.1.2.  OGC API — Features

The OGC API — Features standard offers the capability to create, modify, and query spatial data on the Web and specifies requirements and recommendations for APIs that want to follow a standard way of sharing feature data. The specification is a multi-part standard. Part 1, labelled the Core, describes the mandatory capabilities that every implementing service has to support and is restricted to read-access to spatial data that is referenced to the World Geodetic System 1984 (WGS 84) Coordinate Reference System (CRS). Part 2 enables the use of different CRS, in addition to the WGS 84. Additional capabilities that address specific needs will be specified in additional parts. Envisaged future capabilities include, for example, support for creating and modifying data, more complex data models, and richer queries.

The OGC API — Features standard is part of the OGC API family of standards. OGC API standards define modular API building blocks to spatially enable Web APIs in a consistent way. The standards make use of the OpenAPI specification for defining the API building blocks.

4.1.3.  OGC Styles and Symbology

The OGC Symbology Conceptual Model: Core Part Standard (OGC 18-067r3), also known as OGC SymCore, specifies the conceptual basis to define symbology rules for the portrayal of geographical data. It is modular and extensible (one core model, many extensions), also encoding agnostic (one symbology model, many encodings). It contains a minimal set of abstract classes representing explicit extension points of the model.

OGC Styles and Symbology is a vision for a standard that implements the OGC Symbology Conceptual Model: Core Part Standard (OGC 18-067r3). The Web Mapping Code Sprint was a key moment to consolidate the roadmap to achieve this vision. The plan towards such a candidate SymCore 2.0 Standard consists of the definition of a conceptual and logical model, Cascading Style Sheets (CSS) and JavaScript Object Notation (JSON) encodings, as well as a mapping to SLD/SE (eventually with existing well known vendor options). The requirements described will be supported by illustrated and encoded cartographic use cases.

4.1.4.  OGC GeoXACML

GeoXACML defines a geo-specific extension to the XACML Policy Language, as defined by the OASIS standard “eXtensible Access Control Markup Language (XACML), that allows managing access to geographic information and services in an interoperable way across jurisdictions. Although XACML is meant to be used as a multi-purpose language, XACML does not have the capabilities to express geo-specific constraints on access rights. GeoXACML provides support for spatial data types and spatial authorization decision functions. Those data types and functions can be used to define additional spatial constraints for XACML based policies. GeoXACML defines extensions for the policy encoding schema only, and thus does not affect the XACML context schema.

Implementations of OGC API Standards are designed to work over HTTP(S). Securing the deployed APIs is not of concern to the OGC API Standards. However, the requirement to describe the API’s endpoint via OpenAPI introduces standard options to enable authentication. Knowing the user that accesses an API is an important aspect for many use cases. Also, user identification is often used to control access to the data, served by the API. But, no OGC API Standard provides guidance how to apply access control. This leads to an interoperability issue if access is denied, but there is no common (standardized) way to “tell the reason”. OGC’s GeoXACML 3.0 (currently still draft) Standard supports the interoperable exchange of access conditions as policies. GeoXACML 3.0 defines the geospatial extension to OASIS XACML 3.0 and thereby extends the ability to include Attribute Based Access Control also on geometries.

The challenge for the GeoXACML part of the code sprint was to develop a Policy Enforcement Point (PEP) that intercepts Web Feature Service (WFS) 2.0 requests and modifies the request according to an access decision returned by a GeoXACML 3.0 policy. The access control use case was the following: “WFS GetFeature requests shall exclude the NYC Central Park for feature type poly_landmarks”.

4.2.  Candidate OGC Standards

4.2.1.  OGC GeoPose

OGC GeoPose is an OGC Implementation Standard for exchanging the location and orientation of real or virtual geometric objects (Poses) within reference frames anchored to the earth’s surface (Geo) or within other astronomical coordinate systems. GeoPose specifies a single encoding in JSON format (IETF RFC 8259). The use cases addressed by GeoPose involve interactions between information systems or between an information system and a storage medium. The role of a GeoPose is to convey the orientation and position of a real or virtual object.

4.2.2.  OGC API — Records

OGC API — Records provides discovery and access to metadata records about resources such as features, coverages, tiles / maps, models, assets, services or widgets. The draft specification enables the discovery of geospatial resources by standardizing the way collections of descriptive information about the resources (metadata) are exposed. The draft specification also enables the discovery and sharing of related resources that may be referenced from geospatial resources or their metadata by standardizing the way all kinds of records are exposed and managed. The draft OGC API – Records specification is part of the OGC API family of standards.

4.2.3.  OGC GeoParquet

The Apache Parquet project provides an open source, columnar file format that supports both data storage and retrieval. The OGC GeoParquet specification defines how geospatial data should be stored in Parquet format, including the representation of geometries and the required additional metadata.

4.3.  Other Specifications

4.3.1.  AsyncAPI

AsyncAPI is a specification, by the Linux Foundation, for describing and documenting message-driven APIs in a machine-readable and protocol-agnostic format. The specification defines a set of files required to describe a message-driven API. The files describing the such an API in accordance with the AsyncAPI Specification may be encoded in JSON or YAML.

4.3.2.  STAC API

The SpatioTemporal Asset Catalog (STAC) is a specification that offers a language for describing geospatial information, so it can be worked with, indexed, and discovered. The STAC API offers an interface that implements OGC API — Features. Although STAC has been developed outside of the OGC, in the long term it is envisaged that the STAC API specification will be developed into an OGC Community Standard that implements OGC API building blocks that are relevant for the STAC use cases.

4.4.  ASF Apache Projects

4.4.1.  Apache Arrow

TBA

4.4.2.  Apache Baremaps

TBA

4.4.3.  Apache Sedona

Apache Sedona is a high-performance cluster computing system designed specifically for processing large-scale spatial data. The system leverages a number of open-source libraries to execute spatial operations such as Coordinate Reference System (CRS) transformation and file reading.

4.4.4.  Apache SIS

Apache Spatial Information System (SIS) is a free and open source software library for developing geospatial applications. The library is an implementation of OGC GeoAPI 3.0.1 interfaces and can be used for desktop or server applications. Services provided by Apache SIS include metadata, coordinate transformations, filtering and grid coverages. The library is implemented using the Java programming language.

In this code sprint, Apache SIS was used for testing the Geospatial Integrity of Geoscience Software (GIGS) tests runner.

4.5.  OSGeo Projects

4.5.1.  OSGeo GeoNetwork

GeoNetwork is a catalog application for managing spatially referenced resources. It provides metadata editing and search functions as well as an interactive web map viewer.

4.5.2.  OSGeo GeoServer

GeoServer is a Java-based software server that allows users to view and edit geospatial data. Using open standards by the OGC, GeoServer allows for great flexibility in map creation and data sharing.

4.5.3.  OSGeo OpenLayers

OpenLayers is a library for developing browser-based mapping applications. It works with vector and raster data from a variety of sources and is able to reproject data for rendering. The library has broad support for OGC protocols and formats, including WMS, WMTS, Web Feature Service (WFS), Well Known Text (WKT), Well Known Binary (WKB), Geography Markup Language (GML), GeoTIFF/COG, and KML. In addition, it has support for community and non-OGC standards such as GeoJSON, XYZ tiles, TileJSON, and more. OpenLayers is written in JavaScript and is available under the BSD 2-Clause license.

The ol/source/OGCMapTile module provides a source for map tiles from an OGC API – Tiles implementation. The ol/source/OGCVectorTile module provides a source for vector tiles from an OGC API — Tiles implementation.

4.5.4.  OSGeo OWSLib

OWSLib is a Python client for OGC Web Services and their related content models. The project is an OSGeo Community project and is released under a BSD 3-Clause License.

OWSLib supports numerous OGC standards, including increasing support for the OGC API suite of standards. The official documentation provides an overview of all supported standards.

4.5.5.  OSGeo pycsw

pycsw is an OGC API — Records and OGC CSW server implementation written in Python. Started in 2010 (more formally announced in 2011), pycsw allows for the publishing and discovery of geospatial metadata via numerous APIs (CSW 2/CSW 3, OpenSearch, OAI-PMH, SRU), providing a standards-based metadata and catalogue component of spatial data infrastructures. pycsw is Open Source, released under an MIT license, and runs on all major platforms (Windows, Linux, Mac OS X). pycsw is an official OSGeo Project as well as an OGC Reference Implementation.

pycsw supports numerous metadata content and API standards, including OGC API — Records — Part 1.0: Core and its associated specifications. The official documentation provides an overview of all supported standards.

4.5.6.  OSGeo pygeoapi

pygeoapi is a Python server implementation of the OGC API suite of Standards. The project emerged as part of the next generation OGC API efforts in 2018 and provides the capability for organizations to deploy a RESTful OGC API endpoint using OpenAPI, GeoJSON, and HTML. pygeoapi is open source and released under an MIT license. pygeoapi is an official OSGeo Project as well as an OGC Reference Implementation.

pygeoapi supports numerous OGC API Standards. The official documentation provides an overview of all supported standards.

4.5.7.  OSGeo QGIS

QGIS is a free and open-source cross-platform desktop GIS that supports viewing, editing, and analysis of geospatial data.

4.6.  Community Open Source Projects

4.6.1.  OSGeo ZOO-Project

ZOO-Project is a Web Processing Service (WPS) implementation written in C. It is an open source platform which implements the WPS 1.0.0 and WPS 2.0.0 OGC Standards.

4.6.2.  TEAM Engine

The Test, Evaluation, And Measurement (TEAM) Engine is a testing facility that executes test suites developed using the TestNG framework or the OGC Compliance Test Language (CTL). It is typically used to verify specification compliance and is the official test harness of the OGC Compliance Testing Program (CITE).

4.6.3.  CesiumJS

TBA

4.6.4.  OSGeo Leaflet

Leaflet is an open-source JavaScript library for mobile-friendly interactive maps. It works across all major desktop and mobile platforms, can be extended with a variety of plugins, and offers a well-documented API.

4.6.5.  Camptocamp ogc-client

TBA

4.6.6.  HomeAssistant-SensorThings

Home Assistant is an award-winning free/libre/open-source home automation hub software written in Python. It can interface with over a thousand kinds of IoT devices and services, with a focus on domestic environments and data privacy.

HomeAssistant-SensorThings is a client for the OGC SensorThings standard (part 1) in the form of a Home Assistant plugin (or “integration” in Home Assistant jargon). It implements discovery of available sensors in an endpoint, and periodic updates of the values of those sensors.

4.6.7.  Maplibre

TBA

4.6.8.  MDME

TBA

4.6.9.  OL-Cesium

TBA

4.6.10.  Planet Labs go-ogc

TBA

4.6.11.  OSGeo pygeometa

pygeometa provides a lightweight and Pythonic approach for users to easily create geospatial metadata in standards-based formats using simple configuration files (affectionately called metadata control files (MCF)). The software has minimal dependencies (install is less than 50 kB), and provides a flexible extension mechanism leveraging the Jinja2 templating system. Leveraging the simple but powerful YAML format, pygeometa can generate metadata in numerous standards. Users can also create their own custom metadata formats which can be plugged into pygeometa for custom metadata format output. pygeometa is open source and released under an MIT license.

For developers, pygeometa provides a Pythonic API that allows developers to tightly couple metadata generation within their systems and integrate nicely into metadata production pipelines.

The project supports various metadata formats out of the box including ISO 19115, the WMO Core Metadata Profile, and the WIGOS Metadata Standard. The project also supports the OGC API — Records core record model as well as STAC (Item).

4.6.12.  Geomapfish

TBA

4.6.13.  Smapshot

Smapshot is a crowdsourcing platform allowing volunteers to geolocalise in 3D historical images. More than 1’000 volunteers have geolocalised up to 250’000 images. An Open API is available to integrate the data: https://smapshot.heig-vd.ch/development_documentation and the API and georeferencer are open source

5.  Results

The code sprint included multiple software libraries, OWS implementations, OGC API implementations and different client applications. In addition to supporting OWS and OGC API standards, various ASF and OSGeo software products involved in the code sprint also supported a variety of OGC encoding standards. This section presents some of the results from the code sprint.

5.1.  Approved OGC Standards

5.1.1.  OGC SensorThings API

TBA

5.1.2.  OGC API — Features

TBA

5.1.3.  OGC Styles and Symbology

TBA

5.1.4.  OGC GeoXACML

5.1.4.1.  Purpose

Secure Dimensions implemented the GeoXACML 3.0 draft specification based on the Authzforce CE project which is an Open Source Community Edition project. The implementation is based on Java 11 and shall be made available as the Open Source reference implementation for GeoXACML 3.0. The implementation of a PEP during the Codesprint is important to test the GeoXACML 3.0 GeoPDP reference implementation by Secure Dimensions regarding request and response processing.

5.1.4.2.  Proposed Solution

With respect to implementation, a Geoserver deployment including the default data will be used as the API that is to be protected. From the Geoserver architecture, different possibilities exist where to place the PEP. For the Codesprint, the solution to deploy a Tomcat Filter is favored. This ensures that the Geoserver installation can almost remain unchanged. The configuration of the “web.xml” must just be adopted to include the “WFS Filter for GeoXACML 3.0”. Also, the filter’s JAR and dependencies must be added to “geoserver.war”.

The implementation of the WFS Filter itself will use the Authzforce XACML-SDK for Java that needs to be extended to allow the inclusion of Geometry attributes. The actual access condition to exclude all features of the Central Park should be modelled in a GeoXACML 3.0 Policy. This is possible by constructing an Obligation on Permit that returns the OGC Filter condition that the PEP needs to apply to the intercepted request.

5.1.4.3.  Achievement

During the Code Sprint, a demo was implemented that follows the XACML flow diagram.

image

Figure 2 — XACML Flow Diagram

A Geoserver 2.23 WAR-file deployment was used create the WFS 2.0 service endpoint.

The injection of the PEP was done by updating the “geoserver.war” file manually

  • zip -u geserver.war WEB-INF/web.xml

  • zip -u geoserver.war WEB-INF/lib/GeoXACMLFilter.jar

The web.xml file contains the deployment for the WFS Filter for GeoXACML 3.0.

<filter>
<filter-name>GeoXACMLFilter</filter-name>
<filter-class>de.securedimensions.geoxacml3.ows.GeoServerFilter</filter-class>
<init-param>
<param-name>pdpURL</param-name>
<param-value>https://ogc.demo.secure-dimensions.de/authzforce-ce</param-value>
</init-param>
<init-param>
<param-name>pdpDomain</param-name>
<param-value>A0bdIbmGEeWhFwcKrC9gSQ</param-value>
</init-param>
</filter>
<filter-mapping>
<filter-name>Set Character Encoding</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
<!-- Uncomment following filter to enable CORS
<filter-mapping>
<filter-name>cross-origin</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
-->

<filter-mapping>
<filter-name>GeoXACMLFilter</filter-name>
<url-pattern>/wfs</url-pattern>
</filter-mapping>

Geoserver web.xml initializing the WFS Filter for GeoXACML 3.0 on path /wfs

During the Codesprint one bug concerning the processing of XML attributes was fixed.

5.1.4.4.  Demonstration

During the Code Sprint, the focus was to demonstrate the ability to rewrite WFS requests to reflect geospatial access control decisions. In order to achieve that, a simple GeoXACML 3.0 policy was crafted using ALFA for Visual Studio Code (ALFA plugin).

namespace ogc_codesprint {
    import GeoXACML.*
    import Attributes.*

    obligation Filter = "urn:secd:wfs:filter"

    policyset Manhattan = "root" {
        target clause service == "WFS"
        apply permitOverrides
        policy POLY_LANDMARKS = "poly_landmarks" {
            target clause feature_type == "tiger:poly_landmarks"
            apply firstApplicable
            rule CENTRAL_PARK {
                target clause request == "GetFeature"
                permit
                condition(geometryOneAndOnly(bbox) >< "POLYGON((40.767774 -73.981464, 40.768268 -73.981396, 40.768483 -73.981634, 40.769272 -73.981131, 40.76983 -73.980652, 40.770488 -73.980215, 40.771096 -73.9798, 40.771753 -73.979298, 40.77241 -73.978862, 40.773018 -73.978447, 40.773708 -73.977923, 40.7743 -73.977465, 40.774859 -73.977072, 40.775565 -73.97657, 40.776288 -73.97609, 40.776947 -73.975588, 40.777554 -73.97513, 40.778244 -73.974671, 40.778852 -73.974234, 40.779427 -73.973776, 40.782105 -73.971899, 40.782795 -73.971375, 40.78342 -73.970917, 40.783995 -73.970437, 40.784685 -73.969956, 40.785293 -73.969498, 40.785983 -73.969018, 40.786624 -73.968537, 40.787264 -73.968036, 40.787922 -73.96762, 40.78848 -73.967119, 40.789105 -73.966704, 40.789713 -73.966245, 40.790353 -73.965787, 40.79106 -73.965241, 40.791684 -73.964804, 40.792358 -73.964281, 40.794247 -73.962905, 40.794905 -73.962403, 40.795545 -73.961966, 40.796153 -73.961529, 40.796761 -73.961049, 40.797418 -73.960612, 40.798125 -73.960109, 40.798782 -73.959607, 40.799374 -73.959149, 40.800047 -73.95869, 40.800425 -73.958428, 40.800507 -73.958124, 40.800588 -73.957885, 40.799509 -73.955312, 40.798298 -73.95248, 40.797003 -73.94954, 40.79669 -73.94952, 40.796329 -73.949761, 40.795705 -73.950241, 40.795031 -73.950744, 40.794374 -73.951159, 40.793684 -73.95177, 40.79306 -73.952142, 40.792419 -73.9526, 40.791729 -73.953103, 40.791154 -73.953496, 40.790414 -73.954042, 40.789199 -73.954937, 40.788624 -73.955352, 40.78795 -73.955854, 40.78726 -73.956335, 40.786669 -73.956815, 40.786028 -73.957339, 40.78542 -73.957732, 40.784796 -73.958212, 40.784188 -73.958627, 40.783514 -73.959086, 40.782873 -73.959588, 40.782233 -73.96009, 40.781625 -73.960548, 40.780852 -73.961029, 40.780294 -73.961466, 40.779587 -73.961946, 40.779012 -73.962383, 40.778388 -73.962863, 40.777747 -73.963343, 40.777106 -73.963845, 40.776334 -73.964391, 40.775726 -73.964871, 40.77502 -73.965438, 40.774494 -73.965939, 40.773771 -73.966398, 40.773196 -73.966856, 40.772523 -73.967315, 40.7718 -73.967817, 40.771225 -73.968253, 40.770585 -73.96869, 40.769992 -73.969148, 40.769368 -73.969607, 40.76871 -73.970065, 40.768135 -73.970501, 40.767511 -73.970981, 40.766837 -73.971397, 40.766213 -73.971898, 40.765605 -73.972313, 40.764981 -73.972793, 40.764389 -73.973251, 40.764621 -73.973791, 40.765651 -73.976428, 40.766812 -73.97926, 40.767575 -73.981008, 40.767774 -73.981464))":geometry)
                on permit {
                    obligation Filter {
                        Attributes.Filter.operation = "Disjoint"
                        Attributes.Filter.geometry = "POLYGON((40.767774 -73.981464, 40.768268 -73.981396, 40.768483 -73.981634, 40.769272 -73.981131, 40.76983 -73.980652, 40.770488 -73.980215, 40.771096 -73.9798, 40.771753 -73.979298, 40.77241 -73.978862, 40.773018 -73.978447, 40.773708 -73.977923, 40.7743 -73.977465, 40.774859 -73.977072, 40.775565 -73.97657, 40.776288 -73.97609, 40.776947 -73.975588, 40.777554 -73.97513, 40.778244 -73.974671, 40.778852 -73.974234, 40.779427 -73.973776, 40.782105 -73.971899, 40.782795 -73.971375, 40.78342 -73.970917, 40.783995 -73.970437, 40.784685 -73.969956, 40.785293 -73.969498, 40.785983 -73.969018, 40.786624 -73.968537, 40.787264 -73.968036, 40.787922 -73.96762, 40.78848 -73.967119, 40.789105 -73.966704, 40.789713 -73.966245, 40.790353 -73.965787, 40.79106 -73.965241, 40.791684 -73.964804, 40.792358 -73.964281, 40.794247 -73.962905, 40.794905 -73.962403, 40.795545 -73.961966, 40.796153 -73.961529, 40.796761 -73.961049, 40.797418 -73.960612, 40.798125 -73.960109, 40.798782 -73.959607, 40.799374 -73.959149, 40.800047 -73.95869, 40.800425 -73.958428, 40.800507 -73.958124, 40.800588 -73.957885, 40.799509 -73.955312, 40.798298 -73.95248, 40.797003 -73.94954, 40.79669 -73.94952, 40.796329 -73.949761, 40.795705 -73.950241, 40.795031 -73.950744, 40.794374 -73.951159, 40.793684 -73.95177, 40.79306 -73.952142, 40.792419 -73.9526, 40.791729 -73.953103, 40.791154 -73.953496, 40.790414 -73.954042, 40.789199 -73.954937, 40.788624 -73.955352, 40.78795 -73.955854, 40.78726 -73.956335, 40.786669 -73.956815, 40.786028 -73.957339, 40.78542 -73.957732, 40.784796 -73.958212, 40.784188 -73.958627, 40.783514 -73.959086, 40.782873 -73.959588, 40.782233 -73.96009, 40.781625 -73.960548, 40.780852 -73.961029, 40.780294 -73.961466, 40.779587 -73.961946, 40.779012 -73.962383, 40.778388 -73.962863, 40.777747 -73.963343, 40.777106 -73.963845, 40.776334 -73.964391, 40.775726 -73.964871, 40.77502 -73.965438, 40.774494 -73.965939, 40.773771 -73.966398, 40.773196 -73.966856, 40.772523 -73.967315, 40.7718 -73.967817, 40.771225 -73.968253, 40.770585 -73.96869, 40.769992 -73.969148, 40.769368 -73.969607, 40.76871 -73.970065, 40.768135 -73.970501, 40.767511 -73.970981, 40.766837 -73.971397, 40.766213 -73.971898, 40.765605 -73.972313, 40.764981 -73.972793, 40.764389 -73.973251, 40.764621 -73.973791, 40.765651 -73.976428, 40.766812 -73.97926, 40.767575 -73.981008, 40.767774 -73.981464))":geometry
                    }
                }
            }
            rule ALL_PERMIT {
                permit
            }
        }

    }
}

GeoXACML 3.0 PolicySet described in ALFA

The GeoXACML 3.0 policy is structured in a simple way:

  • PolicySet (Manhattan) matches “service == WFS”

  • Policy (POLY_LANDMARKS) matches “typesNames == tiger:poly_landmarks”

  • Rule (CENTRAL_PARK) matches “request == GetFeature”

  • The Rule Condition contains is geospatial “BBOX Intersects Polygon(…)”

Any request that matches the condition results in the decision “Permit” with the Obligation “urn:secd:filter”. As specified in the XACML 3.0 specification, a PEP must enforce the decision including all obligations. The processing of this filter obligation provides the missing information to construct the WFS Filter (disjoint Central Park).

The result of this processing can be visualized with QGIS (WFS Layer):

image

Figure 3 — Left: Feature type “poly_landmarks” without PEP → Central Park feature(s) are included; Right: Feature type “poly_landmarks” with PEP → Central Park feature(s) are excluded!

The implementation of the Filter obtains the information from the HTTP request:

SubjectCategory subjectCat = new SubjectCategory();
ResourceCategory resourceCat = new ResourceCategory();
ActionCategory actionCategory = new ActionCategory();
EnvironmentCategory environmentCategory = new EnvironmentCategory();

AttributeValueType serviceType = new AttributeValueType(Arrays.asList(httpRequest.getParameter("SERVICE")), XACMLDatatypeId.STRING.value(), null);
Attribute service = new Attribute(Arrays.asList(serviceType),"urn:ogc:ows:service", "", false);
resourceCat.addAttribute(service);

Sample code for obtaining information from the HTTP request

Using the XACML-SDK for Java from Authzforce, the response from the PDP can be obtained in a few lines of code:

Request xacmlRequest = Utils.createXacmlRequest(Arrays.asList(subjectCat), Arrays.asList(resourceCat), Arrays.asList(actionCategory), Arrays.asList(environmentCategory));

ResponsesFactory xacmlResponse = pdp.getAuthZ(subjectCat, resourceCat, actionCategory, environmentCategory);
for (Response r : xacmlResponse.getResponses()) {
    LOGGER.info("XACML Response: " + r.toString());
    DecisionType decision = r.getDecision();
    LOGGER.info("XACML Decision: " + decision.toString());
    LOGGER.info("decision: " + decision.value());
    for (Obligation obligation : r.getObligations().getObligations()) {
        if (obligation.getObligationId().equalsIgnoreCase("urn:secd:wfs:filter")) {
            for (AttributeAssignment aa : obligation.getAttributeAssignments()) {
                if (aa.getAttributeId().equalsIgnoreCase("urn:secd:filter:geometry")) {
                    filterGeometry = aa.getContent().get(0).toString();
                }
                if (aa.getAttributeId().equalsIgnoreCase("urn:secd:filter:operation")) {
                    filterOperation = aa.getContent().get(0).toString();
                }
            }

        }
    }
}

Sample code for obtaining the response from the PDP

5.1.4.5.  Lessons Learned

The Tomcat Filter implementation was based on Java 11. The existing XACML-SDK for Java was available for Java 8. Due to deprecation of javax classes in Java 11, JAXB related functionality had to be updated. The use of GeoTools to create the Filter programmatically could not be achieved. The unresolved problem was that the XML encoder did not include the CRS into the GML part of the spatial filter. Examples and documentation found did only cover non-spatial examples or BBOX Filter. But for implementing the use case, a Disjoint filter with a GML3 geometry had to be constructed. After removing GeoTools completely, a simple string template was used: “<fes:OPERATION><fes:ValueReference>the_geom</fes:ValueReference>GEOMETRY</fes: OPERATION >” where the GEOMETRY was constructed from the response by the PDP (Obligation attributes urn:secd:filter:geometry and urn:secd:filter:operation).

<?xml version='1.0' encoding='UTF-8'?><ns4:Response xmlns:ns6="http://authzforce.github.io/pap-dao-flat-file/xmlns/properties/3.6" xmlns:ns5="http://authzforce.github.io/core/xmlns/pdp/8" xmlns:ns4="urn:oasis:names:tc:xacml:3.0:core:schema:wd-17" xmlns:ns3="http://www.w3.org/2005/Atom" xmlns:ns2="http://authzforce.github.io/rest-api-model/xmlns/authz/5"><ns4:Result><ns4:Decision>Permit</ns4:Decision><ns4:Obligations><ns4:Obligation ObligationId="urn:secd:wfs:filter"><ns4:AttributeAssignment AttributeId="urn:secd:filter:operation" Category="urn:oasis:names:tc:xacml:3.0:attribute-category:resource" DataType="http://www.w3.org/2001/XMLSchema#string">Disjoint</ns4:AttributeAssignment><ns4:AttributeAssignment AttributeId="urn:secd:filter:geometry" Category="urn:oasis:names:tc:xacml:3.0:attribute-category:resource" DataType="urn:ogc:def:geoxacml:3.0:data-type:geometry">POLYGON ((40.767774 -73.981464, 40.768268 -73.981396, 40.768483 -73.981634, 40.769272 -73.981131, 40.76983 -73.980652, 40.770488 -73.980215, 40.771096 -73.9798, 40.771753 -73.979298, 40.77241 -73.978862, 40.773018 -73.978447, 40.773708 -73.977923, 40.7743 -73.977465, 40.774859 -73.977072, 40.775565 -73.97657, 40.776288 -73.97609, 40.776947 -73.975588, 40.777554 -73.97513, 40.778244 -73.974671, 40.778852 -73.974234, 40.779427 -73.973776, 40.782105 -73.971899, 40.782795 -73.971375, 40.78342 -73.970917, 40.783995 -73.970437, 40.784685 -73.969956, 40.785293 -73.969498, 40.785983 -73.969018, 40.786624 -73.968537, 40.787264 -73.968036, 40.787922 -73.96762, 40.78848 -73.967119, 40.789105 -73.966704, 40.789713 -73.966245, 40.790353 -73.965787, 40.79106 -73.965241, 40.791684 -73.964804, 40.792358 -73.964281, 40.794247 -73.962905, 40.794905 -73.962403, 40.795545 -73.961966, 40.796153 -73.961529, 40.796761 -73.961049, 40.797418 -73.960612, 40.798125 -73.960109, 40.798782 -73.959607, 40.799374 -73.959149, 40.800047 -73.95869, 40.800425 -73.958428, 40.800507 -73.958124, 40.800588 -73.957885, 40.799509 -73.955312, 40.798298 -73.95248, 40.797003 -73.94954, 40.79669 -73.94952, 40.796329 -73.949761, 40.795705 -73.950241, 40.795031 -73.950744, 40.794374 -73.951159, 40.793684 -73.95177, 40.79306 -73.952142, 40.792419 -73.9526, 40.791729 -73.953103, 40.791154 -73.953496, 40.790414 -73.954042, 40.789199 -73.954937, 40.788624 -73.955352, 40.78795 -73.955854, 40.78726 -73.956335, 40.786669 -73.956815, 40.786028 -73.957339, 40.78542 -73.957732, 40.784796 -73.958212, 40.784188 -73.958627, 40.783514 -73.959086, 40.782873 -73.959588, 40.782233 -73.96009, 40.781625 -73.960548, 40.780852 -73.961029, 40.780294 -73.961466, 40.779587 -73.961946, 40.779012 -73.962383, 40.778388 -73.962863, 40.777747 -73.963343, 40.777106 -73.963845, 40.776334 -73.964391, 40.775726 -73.964871, 40.77502 -73.965438, 40.774494 -73.965939, 40.773771 -73.966398, 40.773196 -73.966856, 40.772523 -73.967315, 40.7718 -73.967817, 40.771225 -73.968253, 40.770585 -73.96869, 40.769992 -73.969148, 40.769368 -73.969607, 40.76871 -73.970065, 40.768135 -73.970501, 40.767511 -73.970981, 40.766837 -73.971397, 40.766213 -73.971898, 40.765605 -73.972313, 40.764981 -73.972793, 40.764389 -73.973251, 40.764621 -73.973791, 40.765651 -73.976428, 40.766812 -73.97926, 40.767575 -73.981008, 40.767774 -73.981464))</ns4:AttributeAssignment></ns4:Obligation></ns4:Obligations>

Disjoint filter with a geometry constraint

The rewritten Filter was then processed by Geoserver:

typeNames[0] = {http://www.census.gov}poly_landmarks
srsName = urn:ogc:def:crs:EPSG::4326
filter = [[ the_geom within POLYGON ((40.46203574999999 -74.35610985937501, 40.46203574999999 -74.103704953125, 40.674587249999995 -74.103704953125, 40.674587249999995 -74.35610985937501, 40.46203574999999 -74.35610985937501)) ] AND [ the_geom disjoint POLYGON ((40.767774 -73.981464, 40.768268 -73.981396, 40.768483 -73.981634, 40.769272 -73.981131, 40.76983 -73.980652, 40.770488 -73.980215, 40.771096 -73.9798, 40.771753 -73.979298, 40.77241 -73.978862, 40.773018 -73.978447, 40.773708 -73.977923, 40.7743 -73.977465, 40.774859 -73.977072, 40.775565 -73.97657, 40.776288 -73.97609, 40.776947 -73.975588, 40.777554 -73.97513, 40.778244 -73.974671, 40.778852 -73.974234, 40.779427 -73.973776, 40.782105 -73.971899, 40.782795 -73.971375, 40.78342 -73.970917, 40.783995 -73.970437, 40.784685 -73.969956, 40.785293 -73.969498, 40.785983 -73.969018, 40.786624 -73.968537, 40.787264 -73.968036, 40.787922 -73.96762, 40.78848 -73.967119, 40.789105 -73.966704, 40.789713 -73.966245, 40.790353 -73.965787, 40.79106 -73.965241, 40.791684 -73.964804, 40.792358 -73.964281, 40.794247 -73.962905, 40.794905 -73.962403, 40.795545 -73.961966, 40.796153 -73.961529, 40.796761 -73.961049, 40.797418 -73.960612, 40.798125 -73.960109, 40.798782 -73.959607, 40.799374 -73.959149, 40.800047 -73.95869, 40.800425 -73.958428, 40.800507 -73.958124, 40.800588 -73.957885, 40.799509 -73.955312, 40.798298 -73.95248, 40.797003 -73.94954, 40.79669 -73.94952, 40.796329 -73.949761, 40.795705 -73.950241, 40.795031 -73.950744, 40.794374 -73.951159, 40.793684 -73.95177, 40.79306 -73.952142, 40.792419 -73.9526, 40.791729 -73.953103, 40.791154 -73.953496, 40.790414 -73.954042, 40.789199 -73.954937, 40.788624 -73.955352, 40.78795 -73.955854, 40.78726 -73.956335, 40.786669 -73.956815, 40.786028 -73.957339, 40.78542 -73.957732, 40.784796 -73.958212, 40.784188 -73.958627, 40.783514 -73.959086, 40.782873 -73.959588, 40.782233 -73.96009, 40.781625 -73.960548, 40.780852 -73.961029, 40.780294 -73.961466, 40.779587 -73.961946, 40.779012 -73.962383, 40.778388 -73.962863, 40.777747 -73.963343, 40.777106 -73.963845, 40.776334 -73.964391, 40.775726 -73.964871, 40.77502 -73.965438, 40.774494 -73.965939, 40.773771 -73.966398, 40.773196 -73.966856, 40.772523 -73.967315, 40.7718 -73.967817, 40.771225 -73.968253, 40.770585 -73.96869, 40.769992 -73.969148, 40.769368 -73.969607, 40.76871 -73.970065, 40.768135 -73.970501, 40.767511 -73.970981, 40.766837 -73.971397, 40.766213 -73.971898, 40.765605 -73.972313, 40.764981 -73.972793, 40.764389 -73.973251, 40.764621 -73.973791, 40.765651 -73.976428, 40.766812 -73.97926, 40.767575 -73.981008, 40.767774 -73.981464)) ]]

Rewritten Filter

5.1.4.6.  Follow-Up

The OGC API — Features endpoint was not used as it is not yet possible to pass a Filter parameter. Once the Filter capability is standardized in OGC API — Features — Part 3, the implementation for this demonstration could be adopted accordingly. The implementation would need to be adopted to obtain relevant information such as the feature-type, the request BBOX, etc. according to OGC API — Features.

5.1.4.7.  Conclusion

This Codesprint activity demonstrated the basics for enabling powerful Attribute Based Access Control including geospatial conditions. Regarding a production solution, the request rewriting to inject (not only) geospatial conditions via the OGC Filter parameter can be considered high-performance: little information is required to create a request in the PEP and send it to the GeoPDP. Likewise, the response from the GeoPDP to the PEP includes a decision and zero to multiple obligations that control the construction of the Filter. The deployment of the PDP is missing critical, but because this backend service is stateless, scaling is “straight forward” in cloud infrastructures.

A successful demonstration was given during the last day brief back. The QGIS application was used to connect to the GeoServer WFS using the regular OGC WFS 2.0 service. Invisible to the client, the PEP modified the WFS request sent by QGIS based on the access rights exptessed in the GeoXACML policy.

5.2.  Candidate OGC Standards

5.2.1.  OGC GeoPose

TBA

5.2.2.  OGC API — Records

The OGC API — Records team held two breakouts which focused on advancing the specification for OAB review.

Discussions included:

  • extension for faceted searching

  • Record model: contacts and roles

  • STAC harmonization

5.2.3.  OGC GeoParquet

TBA

5.3.  Other Specifications

5.3.1.  AsyncAPI

TBA

5.3.2.  STAC API

TBA

5.4.  ASF Apache Projects

5.4.1.  Apache Arrow

TBA

5.4.2.  Apache Baremaps

TBA

5.4.3.  Apache Sedona

Apache Sedona leverages a number of open-source libraries, including GeoTools, to execute some spatial operations. However, as an Apache project, Sedona is striving to reduce its reliance on GeoTools due to licensing issues. In order to address this challenge, Sedona is migrating affected functions to Apache SIS.

In this code sprint, sedona maintainers concentrated on replacing GeoTools’ usage in two key functions within Sedona: ST_Transform and the GeoTiff reader. Their focus was on demonstrating how to migrate these functions to Apache SIS, providing a comprehensive tutorial on the necessary steps involved.

5.4.4.  Apache SIS

TBA

5.5.  OSGeo Projects

5.5.1.  OSGeo GeoNetwork

TBA

5.5.2.  OSGeo GeoServer

The GeoServer team worked remote focusing on seperating our implementation into individually downloadable modules. This work is required in anticpation of cql2 being finalized enabling the first public release of a GeoServer ogcapi module.

  • Pull-reqeust (merged) splitting ogcapi implementation into individual downloads

  • Pull-request (merged) nightly build docker image allowing public testing of ogcapi modules above

We also checked in with the status of CQL2 which reqmains under active development. We are waiting on this activity before publishing our ogcapi-features module for the general public.

5.5.3.  OSGeo OpenLayers

TBA

5.6.  OWSLib

The OWSLib project updated support for OGC API — Features — Part 4: Create, Replace, Update and Delete, as well as upgraded its public documentation to use Read the Docs as well a continuous deploymnent workflow for automated documentation updates.

5.6.1.  OSGeo pycsw

The pycsw project upgraded support for STAC API to v1.0.0, and added updates/fixes to its support for OGC API — Features — Part 3: Filtering and Common Query Language (CQL2).

5.6.2.  OSGeo pygeoapi

The pygeoapi project completed the implementation of a feature provider for the ERDDAP data server. ERDDAP is a data server that provides a simple, consistent way to download subsets of scientific datasets in common file formats, graphs and maps. This new pygeoaopi feature was created by WMO as part of the https://docs.wis2box.wis.wmo.int project and has been promoted to be included in the pygeoapi core.

Figure 4 — Screenshot of the pygeoapi OGC API - Features provider for the ERDDAP data server.

An update was also made in pygeoapi to highlight licensing when available as part of a collection definition’s link object (where rel=license).

The pygeoapi project implemented the draft OGC API — Pub/Sub conformance class of OGC API — Environmental Data Retrieval standard (see also original discussion paper (23-013)).

Figure 5 — pygeoapi OGC API - Pub/Sub architecture.

Using OWSLib support for OGC API — Features — Part 4: Create, Replace, Update and Delete, the workflow entailed publsihing new features via transactions, which then triggered push notifiations via the MQTT protocol. Pub/Sub capabilities were made available via the AsyncAPI specification, leveraging the OGC API endpoints as subscription channels.

Figure 6 — pygeoapi OGC API - Pub/Sub architecture.

5.6.3.  OSGeo QGIS

TBA

5.7.  Community Open Source Projects

5.7.1.  OSGeo ZOO-Project

The ZOO-Project worked on moving forward OSGeo Incubation as well as the remaining tasks as part of the Project Graduation Checklist to become an OSGeo project.

5.7.2.  TEAM Engine

The aim of the code sprint was to migrate TEAM Engine to Java 17 and to get it running with the OGC API Features test suite. This is an important part of the upcoming TEAM Engine 6.0 release which will focus on updating Java to version 17, Tomcat to version 10 and Maven dependencies including Java libraries and Maven plugins.

First, a test setup was built using a Dockerfile which contains Java 17, Tomcat 7, the latest build of TEAM Engine 6.0-SNAPSHOT and the OGC API Features tests suite compiled with Java 8.

Then, the code of TEAM Engine was migrated to Java 17 with the aim that the complete built is successful. This work could be completed and it was possible to build and compile TEAM Engine with command mvn clean install site -DskipTests using Java 17. All results were documented in issue 511.

Afterward, TEAM Engine compiled with Java 17 was tested with OGC API Features tests suite using the above described test setup. No errors were detected during testing. The TEAM Engine webapp started, a login via Web Browser Interface was possible and a test run with OGC API Features test suite could be executed. The generated report was displayed as expected and test details could be viewed. Thus, the aim of the code sprint was reached.

In the remaining time, the work concentrated on further tasks being important for the TEAM Engine 6.0 release.

Identified open tasks were documented in separate issues:

Finally, all of them were summarized in milestone 6.0 of TEAM Engine.

5.7.3.  CesiumJS

TBA

5.7.4.  OSGeo Leaflet

TBA

5.7.5.  Camptocamp ogc-client

TBA

5.7.6.  HomeAssistant-SensorThings

The SensorThings standard proved approachable enough so that HomeAssistant-SensorThings could be developed within the span of the codesprint, albeit not being feature complete.

Figure 7 — Partial screenshot of a Home Assistant dashboard showing SensorThings observations from the British Geological Survey, from their FROST server.

The SensorThings standard has some traits that proved very helpful for the development and debugging of the client: the usage of JSON over HTTP, and the availability of full REST-like URIs within the metadata offered by the endpoint.

The nomenclature of concepts posed a minor challenge. An OGC Datastream is a HA sensor, and a OGC Thing is a HA device; and an OGC API endpoint becomes a config entry in HA. The concept of an OGC Observation is implicit in the behaviour of a HA sensor. The OGC concepts of Sensor, Location, FeatureOfInterest and ObservedProperty do not have an equivalent on HA. The HA concepts of area, zone, scene and others do not have an OGC equivalent.

Several concerns have been raised:

  • Units of measurement are inconsistent across endpoints. A temperature sensor from USGS uses degC, one from Fraunhofer uses °C, and one from BGS uses C. A human will interpret all those as “degrees celsius”, but this mismatch makes automated data comparisons impossible. The standard recommends using UCUM (Unified Codes for Units of Measure), but none of the SensorThings services explored were following this recommendation.

  • The values for the description and definition of ObservedProperty are unreliable, even though they are mandatory fields. All SensorThings services explored were either using empty strings, or reusing the name of an ObservedProperty as its description and/or definition. No servers produce URIs for the ObservedProperty definitions, as mandated by the standard.

  • Home Assistant focuses on realtime or quasi-realtime data, and the only viable strategy for SensorThings client is to periodically poll for new observations. However, there is no estimation on how often new observations are made, not even a rough one. Some sensors can provide new observations every second, while others might provide observations once every hour, or once per day. Polling data each second from a daily sensor is a waste of networking resources, whereas polling each day from a secondly sensor introduces an obvious data lag. It’s impossible to choose the right approximate rate of polling. The MQTT extension of the SensorThings specification alleviates this issue, but complicates the implementation of both servers and clients.

  • The Home Assistant architecture relies on communicating the last known state of a sensor/device/service; this is arguably the most common use case for internet-connected sensors. However, SensorThings provides no specific facility for this. Instead, the generic way of fetching any and all historical observations must be used, including a query string like ?$top=1&$orderby=phenomenonType desc. The specification states that «the SensorThings API is designed specifically for the resource-constrained IoT devices and the web developer community», but it can be argued that query languages are ill-suited to be implemented in constrained device.

SensorThings is different than other contemporary OGC APIs (such as Maps, Features or Coverages) in that it doesn’t implement content negotiation. It would be interesting to consider content negotiation as an extension, so the last observation could be fetched in human-readable HTML form, or so a series of ovservations could be downloaded as a machine-readable CSV.

Finally, it would be possible to implement a SensorThings server on top of a Home Assistant installation; the scope of such development falls out of a 3-day codesprint, but could be achieved in a timeframe of several weeks. This would provide a cheap way to provide SensorThings endpoints for domestic-grade sensor platforms such as weather stations or soil moisture sensors.

5.7.7.  Maplibre

TBA

5.7.8.  MDME

TBA

5.7.9.  OL-Cesium

TBA

5.7.10.  Planet Labs go-ogc

TBA

5.7.11.  OSGeo pygeometa

The pygeometa project added support for detailed schema capability reporting, so that a client can be aware of which schemas are supported, and whether those schemas have read and/or write support. This was also implemented as pygeoapi plugin, and made available as part of the pygeoapi public demo. “pygeometa as a service” is used by the live demo of the mdme project. The schema capability integration was added by mdme as part of the sprint.

The MDME project uses vjsf internally to render a web based metadata edit form derived from json schema. A visual edit form for MCF has proven to be usefull in yaml oriented metadata workflows for some user groups. Some json schema concepts such as any-of are not supported by vjsf. During the sprint MDME has been improved to prevent the use of any-of and selector patterns having ‘*’. As mentioned above, import from iso19139 and export to iso19139, stac, and ogcapi:records (based on a transformation service provided by pygeoapi/pygeometa) have been introduced.

5.7.12.  Geomapfish

TBA

5.7.13.  Smapshot

Geopose format has been tested as a way to return position and orientation data of the 3D geolocalized images from smapshot from a standard OGC API features. The Geopose data format has been added to the Open Source NodeJs server of OGC API features: CodeSprint PR. A quick data visualisation has been implemented in Cesium to display the geopose of the camera retrieved from the API with smapshot data and the geopose of the photograph.

6.  Discussion

6.1.  GIGS

The GIGS tests can be found on the gigs.iogp.org website. The participants from Geomatys worked on PROJ-JNI to enable it to execute additional GIGS tests. PROJ is a coordinate transformation software package, built using C/C++ library, that transforms geospatial coordinates from one CRS to another. PROJ-JNI is a Java Native Interface for PROJ.

Figure 8 — Example execution of the GIGS test sofware

Not every software product will support every feature that the GIGS tests specify, so a properties file is provided to enable the user to enable or disable specific aspects of the tests. PROJ-JNI is an implementation of the OGC GeoAPI Standard.

6.2.  Maplibre

The rendering pipleline is generally not exposed to the user. So the participants working on Maplibre experimented with accessing the rendering pipeline. They implemented functionality to export the depth buffer (shown in the top left of Figure 9). The approach offers the ability to create light effects and special environmental effects such as an atmospheric haze (shown in the bottom right of Figure 9). So the participants configured their prototype so that when the viewing point is far away the effect is disabled. The participants also examined other parts of the pipeline so that any object that is rendered is based on the modified depth buffer. This included for example modification of the coordinate buffer (shown in the bottom left of Figure 9) so that tiles at the top of the screen represent a large space. The third aspect that can be modified using this approach is the color of the image, for example to enable a user to adjust the brightness or to create aesthetic visuals (shown in the top right of Figure 9).

Figure 9 — Screenshot of Maplibre demonstration

6.3.  Home Assistant and OGC SensorThings API

HomeAssistant-SensorThings was introduced during the code sprint. The sprint participants implemented an ability for HomeAssistant-SensorThings to automatically and periodically fetch observations from an instance of the OGC SensorThings API. The software was also enabled to offer automatic discovery of ‘Things’ as HomeAssistant devices and ‘Datastreams’ as HomeAssistant sensors.

One of the challenges encountered was that there is some inconsistency in units of measurement between specific implementations of the OGC SensorThings API. This issue is explained further by Figure 10 which shows in Panel 1 an implementation that uses degC, and in Panels 2 and 3 other implementations that use C to represent temperature in degree celsius. It is also worth noting the difference in the values of the definition properties in the implementations shown in Panels 2 and 3. This difference could be mistakenly interpreted to mean that the observed properties are semantically different.

Figure 10 — Differences in Units of measure representation in a sample of SensorThings API instances

Another observation was although the data type of the definition property is supposed to be a URI, a number of the implementations appeared not to specify URIs in this property. The sprint participants therefore recommended the creation and maintenance of a taxonomy of well-known observable properties, including their JSON representations containing URIs. Such a capability could, for example, be implemented as an extension of OGC Rainbow (formerly known as the OGC Definitions Server).

Another observation was that there was no indication of the rate of observation. This means it becomes unclear for some client applications how long they should wait before retrieving the next observation. The absence of an indicator of the rate of observation is partly because SensorThings API uses a publish/subscribe mechanism through MQTT and thus does not rely on polling. However, not all client applications support MQTT. The participants noted that OGC API — Records has a resolution property that helds client applications to determine how often it should request observations.

The sprint participants also noted that there is no shorthand for fetching the last observation. The participants noted that although such a query could be constructed using ODATA query options like ?$top=1&$orderby=phenomenonTime desc, such a capability would have to be implemented with caution because some constrained devices might not be able to support querying of very large data collections.

A content-type negotation mechanism that enables a client application to retrieve content in HTML, JSON or another language was also noted as a potential enhancement for the SensorThings API standard.

6.4.  GeoXAML 3.0

TBA

6.5.  pygeoapi

TBA

6.5.1.  ERDDAP

TBA

6.5.2.  AsyncAPI and MQTT for Publish/Subscribe

TBA

6.6.  Styling in OpenLayers

TBA

6.7.  New Vector Representation with GeoParquet, using Apache Arrow

TBA

6.8.  OGC API support through Go

TBA

6.9.  OpenLayers and CesiumJS

TBA

6.10.  OLCesium in Geomapfish

TBA

6.11.  Summary of Lessons Identified

The following are the immediate lessons identified by the sprint participants:

  • TBA

  • TBA

  • TBA

  • TBA

7.  Conclusions

TBA

7.1.  Future Work

The sprint participants made the following recommendations for future innovation work items:

  • TBA

  • TBA

The sprint participants also made the following recommendations for things that the SWGs should consider:

  • TBA

  • TBA


Annex A
(informative)
Revision History

DateReleaseAuthorPrimary clauses modifiedDescription
2022-03-110.1G. Hobonaallinitial version
2022-04-290.2G. HobonaallUpdated with feedback from participants

Bibliography

[1]  OGC: OGC 21-008: Joint OGC OSGeo ASF Code Sprint 2021 Summary Engineering Report, 2021

[2]  Aleksandar Balaban: OGC 21-019, OGC Testbed-17: Attracting Developers: Lowering the entry barrier for implementing OGC Web APIs. Open Geospatial Consortium (2022). https://docs.ogc.org/per/21-019.html